Network-based services systems have become widely available in recent years. Network-based services systems are typically geographically and logically separate from the client subscribing to the network-based service. Network-based services may include one or more computers configured to be storage resources (e.g., virtual disk drives, cloud storage) and/or virtual compute resources (e.g., cloud computing), for example. The resources in various configurations may be used, for example, by businesses (e.g., on-line retail, web based hosting services, financial services) for backup data storage, records storage, to store product images, process customer requests, and/or other computing tasks. In network-based service systems, the performance of the one or more services in the system may be analyzed to determine client usage profiles, peak usage times or latency between various points of the system, for example. To analyze performance, monitors may be implemented at multiple points in the network-based service and the monitors may be configured to continuously monitor many performance metrics of the network-based system (e.g., CPU utilization, latency to respond to requests, available storage, usage profiles, etc.) The continuous stream of performance data may be further compiled to report the results of the monitoring.
As an example, a network-based services system may be configured as a storage service to provide storage for backup data or web hosting services. One or more clients of the network-based service may store (put) and retrieve (get) data from a network-based storage service. A localized resource (e.g., host computer) of the network-based service may monitor the latency for the puts and gets over time. The performance data generated by the monitoring may be used to analyze network and/or service performance. In addition, the monitored data may be stored for further analysis and/or post-processed to further examine the monitored data.
However, as the number of clients, the amount of performance data, and/or the number of performance metrics monitored grows, significant resources may be required to store the monitored data and the post-processed data (e.g., reports), and/or significant compute resources may be required (e.g., CPU) to process the monitored data. As a result, administrators of network-based services may implement data sampling to reduce the amount of monitored data stored. This may result in inaccuracies in system performance data and affect planning and/or responsiveness to system problems.
While the technology described herein is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the drawings and detailed description thereto are not intended to limit the disclosure to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the present disclosure as defined by the appended claims.